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ABSTRACT 


This  twelve-month  research  study  was  a  small  part  of  a  larger  Air  Force  study  to  fulfill 
Department  of  Defense  directives  to  transition  to  a  digital  environment  for  acquisition 
programs  by  the  end  of  2002.  ^  (Appendix  A)  The  research  study,  which  involved  over 
200  active  and  reserve  jet  engine  aircraft  maintenance  troops  at  installations  nationwide, 
was  designed  to  support  the  Air  Force  strategy  of  “Global  Technology  Deployment”  by 
providing  and  testing  jet  engine  maintenance  technical  order  manuals  in  digital  form,  and 
integrating  the  correct  technology  to  utilize  the  digital  application.  In  addition,  it  speaks 
to  the  cultural  issues  associated  with  incorporating  a  dramatic  technological  change. 

Participants  were  sent  the  latest  version  of  the  F-16  interactive  electronic  technical 
manual  (lETM)  software  to  use  for  a  few  months  prior  to  testing  ’’ruggedized”  hardware. 
They  were  also  sent  a  research  survey,  which  measured  the  utility  of  the  combination  of 
software  and  ruggedized  hardware.  Finally,  depot/vendor  sponsored  training  on  the 
software  and  hardware  was  provided  at  various  installations.  The  study  demonstrated 
the  feasibility,  using  “ruggedized”  hardware  and  advanced  software  combinations  and 
the  potential  of  expanding  the  use  of  other  technologies  into  other  areas  of  military 
logistics.  Initial  recruitment  of  participants  was  facilitated  by  the  Air  Force-wide 
deployment  of  I ETM  software  to  all  F-16  Jet  Engine  Propulsion  Maintenance  Shops. 

The  research  survey  and  interviews  proved  to  be  useful  for  data  collection,  which 
allowed  jet  engine  maintenance  troops  to  express  views  and  opinions  on  issues  relevant 
to  the  overall  Air  Force  Project.  Of  those  participants  that  completed  the  survey,  90% 
indicated  they  agreed  or  strongly  agreed  that  this  was  an  effective  method  of 
maintenance.  The  research  concludes  with  recommendations  for  developing  and 
conducting  future  logistics  technology  research  for  the  military. 
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AIR  FORCE  DIGITAL  TECHNICAL  ORDERS: 
TECHNOLOGIES  AND  APPLICATIONS  FOR  THE  FUTURE 


INTRODUCTION 

The  Air  Force  Technical  Order  (TO)  System  provides  educational  and 
operational  tools  for  supporting  AF-wide  operations  and  maintenance  programs.  The 
system  acquires,  maintains,  stores,  distributes,  and  manages  technical  information, 
instructions,  and  safety  procedures  pertaining  to  AF  equipment.  If  TOs  are  not  kept 
accurate  and  current,  downtime  for  maintenance  increases  and  operational  readiness 
decreases.  Untimely  or  inadequate  technical  information  increases  the  possibility  for 
damage  or  destruction  of  equipment  and  creates  potentially  unsafe  operational  and 
maintenance  environments.^ 

The  current  TO  system  is  primarily  paper-based  and  was  designed  in  the  1940s. 
Digitization  of  these  documents  began  in  FY97  with  the  B-1.  Digitization  extended  to 
some  munitions  and  weapons  systems  (AIM7-9,  AGM-88,  AGM-65,  Aerial  Targets,  War 
Reserve  Material  (WRM)  Ammunition,  AEWS,  GTACS,  and  Range  Threats)  in  FY01.  At 
present,  digitization  has  been  completed  for  the  C-17,  C-5,  C-141,  KC-10,  C-130J,  C-9, 
C-20B,  CV-25.^  In  January,  2002,  the  first  F-16  propulsion  systems  were  fully  digitized 
which  allowed  for  improved  interactivity  through  lETM’s.  The  software  was  fielded  to 
active  and  reserve  Air  Force  installations  to  improve  the  efficiency  and  reliability  of  field 
maintenance  on  the  aircraft.  Current  Air  Force  Logistics  Plans  are  for  lETM’s  to  be 
utilized  for  all  next  generation  aircraft  and  weapons  systems  which  are  currently  under 
development. 

Moving  to  a  completely  digital  environment  would  be  advantageous  for  several 
reasons.  First,  a  digital  environment  would  allow  the  most  accurate  and  up-to-date  TOs 
distributed  electronically  in  the  least  amount  of  time.  Secondly,  there  would  be 
considerable  man-hour  savings  in  posting  changes  and  errors  would  be  virtually 
eliminated.  Wear  and  tear  on  paper  products  would  also  be  eliminated. 


In  addition,  there  would  be  a 
signification  reduction  of  the 
“mobility  footprint”,  when  CDs 
or  web-based  access  could 
replace  paper  during 
deployments.  There  are  1.4 
million  pages  of  technical  data 
to  support  the  F-16  fleet 
worldwide — about  14,000 
pounds  of  paper.''  One  e-tool 
device,  tested  in  this  research, 
weighs  about  eight  pounds  and 
can  hold  the  entire  F-16 
technical  data  set,  saving 
valuable  pallet  space. 

Figure  1  Comparison  of  Paper  vs  Digital  Technical  Order 


BACKGROUND 


At  the  request  of  Air  Force  Reserve  Command  (AFRC)/LGA, 

Robins  AFB,  Georgia,  the  Institute  for  Information  Technology 
Applications  (IITA)  was  asked  to  perform  initial  testing  of  e-tools, 
using  a  particular  “e-reader”  device.  The  goReader  (Figure  2) 
was  selected  to  be  tested  simultaneously  with  an  IITA  classroom 
educational  technology  project.  This  device  was  tested  at  three 
F-16  bases,  Buckley  Air  National  Guard  (ANG),  Shaw  AFB  and 
McEntire  ANG  with  disappointing  results.  Some  of  the  major 
shortfalls  were  lack  of  memory  and  poor  battery  life.  Most 
significant,  however,  was  the  lack  of  durability.  When  moving 
the  reader  device  around,  data  would  frequently  surge  or  be 
completely  lost.  (See  Appendix  B,  IITA  Progress  Report) 

Subsequently,  IITA  recommended  follow-on  research  to  be 
conducted  on  “ruggedized”  electronic  technology  for  digital  TO  use.  AFRC  purchased 
the  Panasonic  CF-28  Toughbooks  for  testing  at  4  AFR  F-16  Bases:  The  944‘^  Fighter 
Wing,  Luke  AFB,  AZ;  the  301 FW,  Carswell  JRB;  the  41 9‘^  FW,  Hill  AFB,  Utah;  and  the 
482"'^  FW,  Homestead  AFB,  Florida.  In  addition,  IITA  purchased  a  Panasonic  MDWD- 
07,  “wearable”  computer  and  Panasonic  contributed  one  CF-34  Toughbook  for  additional 
testing. 


Figure  2  goReader  Device 


I.  METHODS 

Experimental  Design.  The  objectives  of  this  research  were  to  examine  the  software  and 
hardware  associated  with  the  Air  Force  digital  Technical  Order  Program  by  providing  the 
necessary  lETMS  software  and  hardware,  facilitate  training,  and  evaluate  data  use  in  the 
field.  Subsequently,  the  methods  used  in  this  research  included:  Site  Survey,  Written 
Field  Survey,  and  actual  deployment  evaluation  of  the  digital  Technical  Order  software 
and  hardware. 

Agencies  and  Participants.  The  research  required  selection  of  a  sample  populace  of  Jet 
Engine  Maintenance  troops  to  participate  in  testing  the  devices.  Participants  were 
selected  with  input  from  Air  Force  Reserve  Command  (AFRC),  Air  Force  Materiel 
Command  (AFMC),  and  the  lETMS’  project  leads  from  the  Air  Force  Depot  Level 
Maintenance  Program.  Participants  were  both  military  and  civilian,  from  both  active  and 
reserve  forces  and  included  all  ranks  within  the  logistics  field. 

The  primary  companies  responsible  for  the  development  and  programming  of  the  lETM 
software  included  Air  Force  General  Contractors  Lockheed  Martin  and  General  Electric. 

Procedures.  In  November  2001 ,  the  CF-28s  were  fielded  to  the  4  F-1 6  Reserve  Bases. 
In  January  2002,  the  bases  received  the  latest  F-16  engine  lETMS. 

Site  Survey 

From  January  2002  to  May  2002  a  series  of  site  surveys  were  conducted  to  assist  in 
implementing  the  new  technology  and  administer  surveys  to  gather  data  to  determine 
the  effectiveness  of  this  technology.  The  principal  criterion  evaluated  in  the  site  survey 
included: 
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•  The  size  of  the  display 

•  Ruggedization  characteristics  of  the  device 

•  Ease  in  locating  the  power  button  on  the  device 

•  The  layout  of  the  keyboard 

•  Ease  in  using  the  pointing  device 

•  Navigating  through  the  program 

•  Ease  in  entering  data  into  the  device 

•  The  use  of  drop  down  menus 

•  Ease  in  reading  data  while  the  device  was  in  direct  sunlight 

•  The  perceived  portability  of  the  device 

•  Ease  in  entering  data  while  wearing  chemical  resistant  gear 

•  Adaptation  of  the  technicians  to  the  new  technology 


Figure  3  Technicians  using  the  CF-34  and  MDWD-07 


Written  Field  Survey 

A  copy  of  the  actual  research  survey  used  in  the  field  is  attached  as  Appendix  B.  The 
paper  survey  was  designed  to  access  whether  or  not  the  digital  lETMS  were  actually 
utilized,  determine  the  computer  skill  levels  of  the  technicians  and  test  the  adequacy  of 
the  hardware  used,  primarily  the  Panasonic  CF-28  Laptop.  Data  was  obtained  regarding 
the  durability  and  ergonomic  issues  associated  with  the  use  of  the  laptop  computer.  In 
addition,  assessments  were  made  on  the  “comfort  level”  of  the  technicians  with  this 
technology. 

Deployment  Evaluation 

The  actual  deployment  of  the  devices  to  off  station  locations  was  tested  by  both  active 
and  reserve  bases.  Deployment  with  lETMs  digital  Technical  Orders  was  first  attempted 
by  Shaw  Air  Force  Base  in  January  2002  during  a  training  mission.  However,  due  to  the 
continuous  upgrades  taking  place  with  the  software  and  hardware,  printed  backup 
technical  data  was  necessary  to  accomplish  much  of  the  maintenance  during  the 
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deployment  to  ensure  continuity.  In  May  2002,  the  482d  Reserve  wing,  Homestead 
ARS,  was  able  to  successfully  accomplish  a  small  training  deployment  to  Tyndall  AFB 
without  the  need  for  paper  TO  assistance.  The  paper  version  was  available  for  back-up. 


II.  CONDUCTING  THE  RESEARCH 
Subject  Demographics 

The  research  data  collected  was  from  maintenance  troops  assigned  to  each  of  the 
bases  listed  in  Table  1  below.  Participants  were  required  to  attend  training  seminars  on 
the  use  of  the  lETMS  software  and  hardware.  The  total  number  of  subjects  participating 
varied  due  to  both  unit  TDY  schedules  and  individuals  voluntarily  requesting  to  be  added 
or  removed  after  the  research  survey  period  had  begun.  The  decision  to  add  personnel 
was  based  on  whether  or  not  the  subject  had  received  the  Air  Force  lETMs  training. 

Wide  varieties  of  expertise  and  years  on  the  F-16  aircraft  existed  in  the  subject  pool; 
therefore  a  typical  profile  could  not  be  established.  However,  the  following  are 
significant  facts  on  the  participants  which  provide  partial  descriptions: 

1 .  The  average  amount  of  Reserve  experience  working  on  jet  engines  was  1 1 
years. 

2.  The  number  of  subjects  reporting  ability  to  respond  to  the  survey  was  21 8. 

3.  Subjects  were  members  of  the  active  duty  USAF,  USAF  Reserve,  Air  National 
Guard,  and  civilian  Air  Reserve  Technicians. 

4.  While  only  5  bases  were  actually  visited,  subjects  applicable  to  this  study  were 
assigned  to  more  than  15  different  installations.  This  larger  sampling  was 
possible  because  IITA  extended  research  by  including  all  attendees  at  jet 
propulsion  lETMs  training,  whose  attendance  was  made  up  of  technicians  from 
active  and  reserve  installations  nation-wide. 

5.  Motivation  to  use  the  new  technology  varied,  but  was  directly  related  to  the  level 
of  computer  skill  expertise. 

Table  1:  Site-Surveyed  Bases 


Date 

Location 

Number  of  Participants 

Sep  01 

Shaw  AFB 

55 

Feb  02 

Hill  AFB 

83 

Mar  02 

Carswell  ARS 

17 

May  02 

Homestead  ARB 

51 

Jan  02 

Luke  AFB 

12 

The  lETM’s  Software 


The  software  was  developed  by  Lockheed  and  General  Electric  under  contract  with  the 
Air  Force  Logistics  Depot  Maintenance  Program.  Each  contractor  has  Defense 
agreements  with  the  Department  of  the  Air  Force  to  develop  the  digital  jet  engine  TO  on 
the  F-1 10  &  220  engines.  Evaluation  of  the  software  took  place  prior  to  field  deployment 
to  determine  the  best  possible  hardware  to  be  used  with  the  program.  The  instructional 
design  for  each  version  of  the  digital  TO  consisted  of  a  web-based  menu  on  the  left  side 
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of  each  screen  with  the  actual  data  appearing  on  the  right  side  of  the  screen.  Both 
remain  visible  on  every  page  of  the  lesson.  Data  is  largely  available  on  CD-ROM  in 
addition  with  web-server  based  capability. 

Many  recommendations  were  provided  to  the  companies  to  improve  previous  versions. 
For  example,  the  need  for  increased  interactivity  for  maintenance  user  interface  was 
demonstrated.  The  following  table  describes  the  components  of  the  lETMS  software. 

Table  2:  lETM  Software  Translations/Definitions 


MENU 

Section  Definition 

TCTO 

Time  Compliance  Technical  Order 

Work  Package 

WP-  Portion  of  the  TO  that  is  being  worked  at  a  given  time. 

Diagram 

Schematic  technical  drawings  as  used  in  printed  paper  TO’s. 

The  software  was  installed  and  resided  on  each  hardware  device  tested.  Limited  CPU 
capability  on  the  earliest  ruggedized  laptop  models  slowed  the  system.  This  was  of 
particular  concern  with  the  flightline  maintenance  crews  who  needed  timely  information. 

Subjects  were  provided  “touch  screen”  capability  and  a  stylus  pen  with  each  hardware 
device  to  immediately  scroll  or  click  to  required  TO  data.  Additionally,  subjects  required 
actual  access  to  the  unit  Technical  Order  Distribution  Office  in  order  to  assure  the 
correct  version  of  the  CD-ROM  was  in  use  (over  15  different  versions  have  already  been 
produced). 

Five  days  prior  to  each  site  visit,  subjects  were  identified  by  the  unit  propulsion 
superintendent.  This  was  to  ensure  that  each  subject  had  at  least  limited  knowledge  of 
using  the  application  and  computers.  Additionally,  during  the  site  visit  a  demonstration 
period  was  conducted,  followed  by  question  and  answer  session  before  the  research 
period  began.  Subjects  then  received  4  or  more  weeks  to  use  the  new  hardware  and 
software  combination.  The  Depot  made  use  of  the  technology  mandatory  for  each 
installation  undergoing  the  test,  which  helped  facilitate  greater  participation  in  the 
project.  Finally,  technicians  were  administered  the  survey. 

III.  SURVEY  RESULTS 


1.  How  often  do  you  use  the  digital  T.O.  in 
your  current  work  environment? 


Always 

21% 


Often 

28% 


Never 

9% 


Som  etim  es 
12% 


About  Half 
the  Tim e 
30% 


Figure  4:  Survey  Responses 
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Survey  respondents  indicated  that  usage  of  the  digital  TO’s  exceeded  20%  at 
installations  using  lETM’s  software  and  hardware.  An  additional  28%  indicated  that  they 
use  the  system  often,  but  not  all  the  time.  A  majority  of  respondents  indicated  use  of  the 
technologies  at  least  half  the  time.  Because  the  sample  for  this  research  was  primarily 
composed  of  those  trained  and  not  completely  familiar  with  the  digital  lETM’s,  field  data 
may  not  accurately  depict  the  actual  usage  of  the  technology  for  the  entire  force. 


2.  How  often  do  you  use  a  desktop  or  laptop  personal 
computer? 


Always 

40% 


Neversometime4^°^l^"'f‘'^^ 

0%  3% 

24% 


Often 


33% 


Figure  5:  Survey  Responses 

Technology  usage  was  a  key  element  of  this  research  because  Air  Force  wide  IT  usage 
is  generally  lower  in  the  operations  and  maintenance  areas.  In  the  propulsion  engine 
shop  arena,  a  majority  of  respondents  indicated  using  computers  of  any  type  well  over 
half  the  time.  Much  of  this  is  a  result  of  Air  Force  computerized  aircraft  maintenance 
system  (CAMS)  records  as  well  as  mobility  aircraft  data  systems  such  as  “G081”  and 
the  command,  control,  communications,  and  information  processing  system  (C3IPS). 
These  systems,  implemented  over  the  past  10  years,  now  expose  engine  technicians  to 
computers  on  a  more  frequent  basis. 


3.  In  using  the  lETM  digital  TOs  on  your  unit’s 
assigned  “Toughbook”  computer,  do  you  ever  stumble 
with  technology  issues  that  require  additional  training? 

Never  Sometimes 
Always  ^  About  Half 

36%  - ^ 

26% 


Often 

35% 


Figure  6;  Survey  Responses 

A  large  number  of  survey  respondents  indicated  that  lack  of  training  on  usage  of  the 
lETM’s  product  was  an  inhibitor  to  daily  production.  Due  to  cost  constraints,  training  on 
lETM’s  was  conducted  for  just  5%  of  installation  troops.  The  training  model  calls  for 
those  fully  trained  to  become  subject  matter  experts  (SME’s)  who,  in  turn,  train  the 
remaining  troops  in  the  unit.  Results  indicate  that  as  of  the  survey  period,  the  training 
was  still  necessary. 
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4.  How  often  do  you  find  the  need  to  cross- 
reference  the  data  from  the  CD-ROM  digital  TO 
with  an  actual  printed  version  of  the  TO? 


Always 

30% 


O  fte  n 
30% 


Never 
8% 


Som  etim  es 
1  1  % 

About  H  alf 
the  Tim  e 
21  % 


Figure  7:  Survey  Responses 

60%  of  the  maintenance  troops  who  responded  to  the  survey  returned  to  paper  TOs 
primarily  due  to  the  lower  confidence  levels  on  the  accuracy  of  the  lETMS  technical 
order.  Reliability  was  questioned  as  a  result  of  the  numerous  revisions  to  the  software. 
During  the  research  period  7  versions  of  TO’s  and  time  compliance  orders  were  made. 
This  concern  was  forwarded  by  our  research  team  to  the  Oklahoma  City  Air  Logistics 
Center  (OC-ALC)  who  is  responsible  for  certifying  and  validating  data  on  digital  TO’s  for 
the  F-16,  Air  Force  wide.  The  Air  Force  TO  Distribution  office  continues  its  effort  in  this 
area. 


5.  During  the  period  you  have  been  using  digitalTOs, 
have  you  had  any  Information  Technology  Problems  with 
the  "Toughbook”  hardware? 


O  fte  n 
1  6% 


About  Hal 
the  Time 
1  9% 


A  Iw  a  y  s 
8% 


Never 

36% 


Some  tim  e  s 
2  1  % 


Figure  8:  Survey  Responses 

IT  troubles  reported  by  respondents  were  higher  than  expected.  Specific  concerns  were 
battery-life  longevity,  screen  visibility,  size  of  text,  software,  and  connectivity.  While 
many  of  these  issues  were  alleviated  with  the  acquisition  of  the  CF-28,  not  all  were 
abated.  There  were  still  concerns  regarding  the  overall  reliability  of  the  electronic  tools 
when  becoming  a  100%  digital  operation.  In  follow-up  interviews  with  the  test-bed 
location  at  Shaw  AFB,  SC,  those  workstations  that  were  fully  digital  and  utilizing  lETMs 
software  experienced  very  little  or  no  difficulty  when  transitioning  to  100%  digital  lETMS 
use.  Comfort  and  training  levels  of  the  subjects  were  found  to  be  a  greater  concern  than 
the  reliability  of  the  data  and  electronic  hardware. 
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6.  Have  you  had  any  applications  problems 
with  the  software  or  CD-ROM  Data? 


Always 

23% 


N  e  ve  r 

Qo/^  Sometimes 
1  5  % 


O  fte  n 
29% 


About  H  a  If 
the  Time 
2  5% 


Figure  9:  Survey  Responses 


Software  applications  problems  did  exist  with  lETMs  and  the  Computer  Automated 
Maintenance  System  (CAMS)  platforms,  especially  when  operating  on  the  same 
hardware  device.  More  than  50%  indicated  having  some  trouble  with  the  lETMs 
applications  of  multiple  web  browsers,  HTML  Frames,  and  searching.  For  example, 
earlier  hardware  and  software  versions  were  slower  while  the  processing  speed  on  the 
CF-28  was  much  faster  allowing  for  better  use  of  web-based  applications.  Technicians 
who  utilized  both  lETMs  and  CAMS  on  the  same  hardware  device  would  like  to  see  even 
greater  speeds  when  opening  graphic  functions  and  searching  for  TO  maintenance  work 
packages.  In  addition,  the  lETMs  application  can  present  trouble  by  “locking-up”  on 
slower  PC’s  and  hardware  devices. 


IV.  DISCUSSION 

The  data  collected  during  this  research  points  to  the  following  major  findings, 
categorized  by  Hardware,  Software  and  Cultural  issues. 

Hardware  Technology  Issues 

The  primary  hardware  used  was  the  Panasonic  CF-28,  but  3  other  approved  Panasonic 
systems,  the  CF-27,  CF-34,  and  MDWD-07  were  fielded  to  test  hardware  possibilities 
based  on  size,  memory,  portability,  durability,  and  usefulness  with  the  lETMs  application. 
Hardware  is  listed  in  Appendix  D.  Scalability  and  durability  were  the  key  hardware 
technology  issues  tested.  The  goal  was  to  reduce  the  size  of  computer  needed  to 
perform  lETMs  work,  but  still  be  adequate  to  display  the  TO.  The  hardware  tested  met 
scalability  requirements. 
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Ruggedized  durability  features,  which  differentiate 
the  product  line  from  the  typical  PC  or  laptop  device 
available  to  most  consumers,  were  essential  for 
flightline  operations.  Each  hardware  device  tested 
was  engineered  for  industrial  use,  and  designed  using 
Military  MIL-STD-810E  test  procedures  used  by  the 
Department  of  Defense.  Features  of  the  hardware 
included  hardened  polymer  cases  which  could 
accommodate  excessive  weight  pressure  and  prevent 
damage.  A  typical  drop-test  of  the  hardware  involved 
an  8ft  drop  onto  hardened  concrete.  Each  device  tested 
easily  passed  this  test.  In  addition,  the  hardware  was  water,  chemical,  and  moisture 
resistant.  This  characteristic  was  very  important  as  Jet  Engine  Maintenance  facilities  are 
surrounded  by  fuels,  oils,  and  grease.  The  typical  device  tested  by  technicians  would 
get  dirty  and  greasy  within  just  a  few  days  use.  Webbed  rubber  lining  for  keyboards  as 
well  as  gasket  gated  portals  for  the  hardware  helped  maintain  hardware  integrity. 


Figure  10  Technician  using  MDWD-07 


The  screens  on  the  hardware  were  upgraded  during 
this  research  to  include  touch  screen  capability  and 
scratch  resistant  enhancements.  This  was  important 
as  technicians  preferred  to  use  their  hands  or  even 
work  tools  as  opposed  to  a  computer  stylus  when 
scrolling  through  work  documentation.  These  upgrades, 
coupled  with  on  screen  capability  and  visibility  were 
helpful  in  allowing  design  engineers  to  accommodate 
military  needs  in  advancing  tools  from  the  CF-27  to  the 
CF-28  device.  The  most  recent  modification  included 
reducing  the  screen  power  usage  requirements  to  facilitate 
longer  battery  life  for  the  associated  hardware.  Figure  11.  Screen  Comparisons 

Software  Issues 


Software  issues  drastically  improved  throughout  the  research  study  as  newer  versions 
hit  the  field. 

The  early  versions  of  digital  TO’s  were  not  web-based.  A  text  only  version  with  PDF  files 
were  used  which  did  not  allow  for  viewing  diagrams  and  graphs  that  are  essential  to 
engine  maintenance  work. 

Software  development  involved  moving  all  engine  technical  data  to  an  interactive 
electronic  format.  Teams  from  AFMC  and  civilian  software  developers  met  to  discuss 
the  application,  and  in  the  end  a  web-based  interface  was  selected  as  the  most  viable 
method  due  to  troop  familiarity  with  application  such  as  Microsoft  Internet  Explorer. 

Versions  1.0  and  2.0  of  the  lETM’s  software  presented  difficulty  with  data  accuracy, 
processing  speeds,  and  program  execution  due  to  the  use  of  ‘transposed’  TO  data  which 
was  converted  form  actual  printed  text  and  programmed  in  xTML  and  HTML  web-based 
languages.  This  resulted  in  several  delays,  first  in  working  out  programming  errors,  and 
later  in  making  the  electronic  data  as  accurate  as  the  printed  TO  version.  This  process 
required  data  to  be  screened  by  Air  Force  TO  managers. 
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At  the  same  time,  diagrams  and  other  graphics  were  not  drawn  to  scale  when  used  in 
the  web-based  mode.  As  a  result,  the  team  decided  to  create  an  enhanced  graphics 
interface  which  would  allow  for  scalability  of  all  images  allowing  them  to  be  blown  up 
beyond  that  of  the  traditional  paper  TO.  As  a  result,  one  of  the  finer  aspects  of  the 
lETMs  software  as  discussed  by  troops  was  the  ability  to  use  enhanced  images  and 
graphics. 

Later  versions  of  the  software  addressed  many  shortcomings  reported  from  the  field. 
Today’s  lETM’s  allows  for  both  network  and  non-network  connectivity  for  data,  and 
permits  multiple  windows  to  be  opened  at  the  same  time  allowing  for  troops  to  access 
both  the  TO,  Indices,  and  other  electronic  maintenance  applications  on  the  same 
computer. 

Cultural  Issues 


Preliminary  assessment  of  troop  “comfort”  with  the  technology  found  some  initial 
hesitation  with  implementation.  In  some  cases,  the  site  survey  found  researchers 
actually  having  to  remove  the  equipment  from  original  packing.  After  familiarization,  the 
lETMS  software,  specifically  the  web-based  format  was  found  to  be  user-friendly  and  the 
subjects  appreciated  having  everything  they  needed  to  do  their  work,  but  there  were 
several  ‘fears  of  technology’  which  inhibited  some  troops  from  being  solid  survey 
subjects.  For  example,  senior  technicians  were  less  computer  savvy,  while  younger 
technicians  navigated  through  the  software  fairly  easy. 

Virtually  all  subjects  reported  using  the  lETMS  work  packages  for  their  maintenance 
tasks,  both  for  routine  maintenance  and  troubleshooting.  Initially,  senior  troops  would 
locate  a  work  package  digitally,  and  then  migrate  to  paper  based  copies.  However, 
younger  troops,  spent  time  learning  each  subsequent  version  of  lETMs  software,  and 
demonstrated  how  the  software  could  be  used  exclusively  for  engine  maintenance  work. 
Senior  participants  did  not  adapt  well  to  multiple  software  version  upgrades  throughout 
the  research  period.  These  updates,  though  required  by  AFMC,  made  training  on  use  of 
lETMS  software  more  difficult. 

With  continuous  use  of  the  software  application,  several  troops  became  more  adept  at 
using  the  digital  lETMs  application  and  found  that  using  the  computer  search  function 
was  more  efficient  than  paper  based  methods.  This  efficiency  also  stems  from 
eliminating  multiple  trips  to  the  TO  library  for  additional  data. 

Due  to  the  limited  time  period  of  this  research,  combined  with  Air  Forces’  3-5  year  plan 
for  total  force  rollout  of  digital  TO’s,  lower  motivational  levels  were  reported  over  the  test 
period.  However,  the  enthusiastic  response  to  the  future  uses  of  the  technology  was 
received  well.  All  subjects  agree  that  programs  of  this  type  are  a  necessity  and  will  be 
highly  successful  over  the  long-term.  Virtually  all  participants  found  that  TO  information, 
diagrams,  and  data  delivered  in  this  manner  would  help  them  maintain  their  training  and 
skill  levels  in  performance  of  jet  engine  maintenance  tasks.  The  bottom  line: 
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1,  Although  most  technicians  were  comfortable  with  the  technology  and  most  use 
the  computer  on  a  daily  basis,  about  20-25%  used  the  digital  TO  for  maintenance 
work.  Research  on  digital  TO  use  employed  by  aircraft  jet  engine  mechanics 
indicates  that  the  strategy  most  often  employed  by  the  troops  was  frequent  return 
to  the  paper  TO  for  reference  -  using  the  digital  version  first. 

2,  Technicians  were  very  concerned  about  training  on  using  lETM’s  and  expressed 
a  desire  for  further  opportunities  to  better  learn  the  computer  applications  and 
systems. 

3,  Each  successive  version  of  the  software  presented  some  confusion  at  the  unit 
level  as  engine  supervisors  became  increasingly  concerned  regarding  the 
accuracy  of  digital  TO  data  and  the  manner  in  which  it  might  affect  operational 
safety. 

4,  The  age  of  the  respondent  was  a  significant  determinant  as  to  whether  the  user 
felt  comfortable  with  the  electronic  applications. 

5,  Hardware  upgrade  of  the  CF-27  to  CF-28  made  a  tremendous  impact.  Future 
upgrades  to  smaller  machines  such  as  the  CF-34,  or  wearable  IVIDWD-07  may 
present  more  operational  efficiency. 

6,  A  wireless  use  lETMs  maintenance  system  is  definitely  needed  to  better  connect 
back  shop  lETMs  with  the  actual  flightline 


Figure  12  CF-34  and  Paper  TO 

Besides  improved  deployment  during  expeditionary  operations,  current  factors  favoring 
digital  software  and  electronic  hardware  based  Technical  Orders  are  the  increasing 
technical  complexity  of  military  jet  engines.  As  propulsion  systems  become  more 
technologically  advanced,  reliable  electronic  tools  are  becoming  more  and  more 
necessary  in  troubleshooting,  repairing,  and  maintaining  them. 

Another  favorable  aspect  of  digital  TO  systems  are  the  realized  savings  in  materials  and 
training  costs  from  less  reliance  on  printed  materials,  and  improved  economies  of  scale 
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associated  with  training  dollars  previously  spent  to  upgrade  the  technical  skills  and 
experience  levels  of  technicians. 

Less  favorable  to  TO  digitization  is  the  issue  of  Logistical  control  over  maintenance 
assets  and  the  potential  of  having  advanced  engines  subject  to  technical  repair  orders 
that  require  immediate  attention.  Linder  the  current  maintenance  structure,  there  is 
value  to  having  technicians  who  know  where  to  immediately  find  the  TO  data  for  a 
particular  repair  under  given  constraints  for  maintenance  turn  times. 

Without  question,  the  favorable  results  of  this  research  outweigh  the  less  favorable 
aspects,  and  there  is  solid  validation  in  the  survey  data  to  support  the  Air  Force  Logistics 
community’s  move  toward  more  digital  products  for  maintenance  and  operations  use. 


V.  CONCLUSIONS/RECOMMENDATION 

Deploying  the  Digital  TO  lETMs  for  the  logistics  community  is  feasible  and  viable. 

Several  of  the  hardware  devices  tested  worked,  but  the  most  recent  laptop  hardware, 
the  Panasonic  CF-28,  proves  to  be  the  most  efficient  computer  of  choice  for  the 
digitization  program.  This  was  recommended  to  MAJCOM  program  managers  and 
acquisition  is  moving  forward  at  the  present  time. 

It  is  recommended  that  the  Air  Force  and  others  in  the  defense  community  develop 
enhanced  over-the-counter  technologies  with  durable  capabilities  that  also  increase  user 
interface  for  software  and  hardware.  This  continuing  research  is  allowing  review  of  test 
and  evaluation  stages  on  a  wide  array  of  ‘next-generation’  handhelds,  and  alternative 
‘ruggedized’  devices  still  in  development.  There  is  great  value  in  continuing  research  in 
this  area.  Immediately  on  the  horizon  are: 

•  “Wearable”  computers.  These  devices,  tested  in  this  research  only  on  the 
MDWD-07,  are  not  ready  for  Air  Force  wide  deployment.  There  are  still  too 
many  technology  issues  that  inhibit  the  successful  deployment  of  the 
technologies  for  military  use  such  as  peer  to  peer  protocols  that  have  limited 
range  and  require  actual  line  of  sight  for  the  computer  central  processing  unit 
(CPU)  and  wearable  screen  to  function.  This  requirement  does  not  operate 
well  in  the  aircraft  maintenance  and  logistics  environment  due  to  the  number 
of  components  and  equipment  that  often  exist  between  technicians  and  their 
respective  workstation  location.  In  addition,  it  is  recommended  that  larger, 
but  still  portable,  display  screens  be  added  to  Research  and  Development  of 
the  wearable  computer  products  in  order  to  better  accommodate  the 
ergonomic  needs  of  both  shop  and  flightline  maintenance.  Future  testing  will 
involve  further  integration  of  the  wearable  peer  to  peer  improvements  along 
with  PCMA  wireless  technology  in  order  to  introduce  a  viable  'next 
generation'  system. 

• 

•  “Wireless  Data”  Sample  wireless  data  streams  and  the  applicability  to  Air 
Force  maintenance  use  in  conjunction  with  the  technologies  tested  for  this 
research  was  briefly  initiated.  There  is  significant  research  potential  for 
integrating  the  hardware  from  this  research  with  the  wireless  maintenance 
environment. 
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•  “Applying  the  Web.”  Web-based  interfaces  carry  tremendous  potential  for 
military  use.  Applications  that  are  web  compatible  are  more  user  friendly  and 
many  troops  already  recognize  the  functions  of  the  web  interface  from  both 
their  personal  and  professional  PC  use.  Further  movement  into  digitization  of 
tech  manuals  and  other  military  applications  should  include  next  generation 
web  technologies  and  programming  such  as  pop-ups,  animations,  better 
touch  screen  synergies,  and  synonymous  display  concepts. 

In  summary,  it  would  be  beneficial  to: 

1 .  Continue  advancing  use  of  this  digital  software/hardware  capability 

2.  Transition  to  server  and  web-based  TOs  for  instantaneous  updates 

3.  Pursue  wireless  concepts  to  flightline/backshops 

4.  Incorporate  tiger-team  training 

5.  Apply  interactive  concept  to  other  LG  databases,  i.e.,  supply  system 

6.  Continue  to  follow  technology  with  upgrades  to  portable  solutions 

7.  Continue  interactive  transition  to  other  weapon  systems. 

8.  Evaluate  hardware  developed  by  other  companies  such  as  Dolch,  Hewlett 
Packard  and  Dell  which  may  facilitate  next  generation  upgrade 

An  important  next  step  in  this  research  is  to  work  with  MAJCOMs  and  other  services  to 
advance  digitization  of  technical  data  to  various  other  weapons  systems.  The  efficiency, 
cost  savings,  and  real  time  capability  of  the  digital  applications  is  such  that  defense-wide 
research  and  development  in  this  area  are  essential.  Current  technologies  of  both 
hardware  and  software  will  benefit  from  further  expansion  and  newer  versions  that 
employ  newer  technologies.  Each  weapons  system  that  enters  the  digital  era  will  have 
fully  upgradeable  and  timely  changeable  technical  data  which  will  benefit  operations  and 
maintenance  as  well  as  better  facilitates  modifications  to  any  weapons  system. 

Elements  of  this  research  focused  primarily  on  backshop  maintenance.  Another 
important  step  is  to  integrate  flightline  and  backshop  maintenance  communications 
systems  which  will  not  only  encompass  digital  TOs,  but  incorporate  aircraft  status 
tracking,  flightline  parts  requisition,  and  back  shop  communications.  This  multi-faceted 
logistics  management  system  will  feed  maintenance  operations  centers  (MOCs)  with 
accurate  time-critical  information  on  mission  capabilities. 

A  concurrent  formalized  training  program  on  IT  applications  associated  with  digitization 
is  recommended.  Updated  skill  level  requirements  on  IT  development  tools  and 
computer  automated  training  devices  and  programs  will  be  an  essential  part  of  preparing 
the  force  for  next  generation  technologies  to  be  used  in  the  field. 

Another  concurrent  effort  should  be  in  the  monitoring  of  next  generation  hardware  and 
software.  Continued  researched  in  to  multiple  displays  with  single  server  access,  as  well 
as  ergonomic  and  technological  improvements  is  essential  in  improving  the  digitization 
process.  Web  enablement,  multi-functions  tools,  portability,  and  durability,  are  but  a  few 
of  the  additional  issues  that  will  be  addressed  in  continuing  research  for  digitization  of 
defense  weapons  systems. 
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Appendix  A 


THE  DEPUTY  SECRETARY  OF  DEFENSE 
WASHINOTON,  O  C.  Z030I-1000 


JUL  Z 


MEMORANDUM  FOR  SECRETARIES  OF  THE  MILITARY  DEPARTOENTS 
CHAIRMAN,  JOINT  CHIEFS  OF  STAFF 
UNDER  SECRETARIES  OF  DEFENSE 
DIRECTOR,  DEFENSE  RESEARCH  AND  ENGINEERING 
ASSISTANT  SECRETARIES  OF  DEFENSE 
DIRECTOR,  OPERATIONAL  TEST  AND  EVALUATION 
DIRECTORS,  DEFENSE  AGENCIES 

SUBJECT:  Policy  for  the  Transition  to  a  Digital  Environaent  for 
Acquisition  Programs 


The  Department  has  made  stdostantial  progress  in  the 
acquisition,  management,  and  use  of  digitized  information.  It  is 
now  time  to  move  forward  to  a  fully  digital  environment  in  all 
acquisition  program  and  support  offices.  Industry  has  already 
demonstrated  that  this  is  not  only  possible,  but  preferable  to 
traditional  paper-driven  systems.  I  am  setting  a  corporate  goal  of 
digital  operations  being  the  method  of  choice  for  all  acquisition 
management  and  life  cycle  support  information.  By  the  end  of  2002, 
the  overwhelming  majority  of  DoD  acquisition  and  logistics 
operations  should  be  based  on  digital  methodologies  and  products. 

The  focus  of  this  effort  mist  be  at  the  program  office  level. 
Consistent  with  the  architecture  established  by  the  joint  DoD  level 
executive  steering  group.  Program  Managers  shall  be  responsible  for 
establishing  a  data  management  system  and  appropriate  digital 
environment  that  allows  every  activity  involved  with  the  program 
throughout  its  total  life-cycle  to  exchange  data  digitally. 

I  am  counting  on  your  support  for  this  critical  initiative 
that  will  enhance  acquisition  reform,  further  empower  our 
Integrated  Product  Teams,  and  combine  with  electronic  commerce  to 
achieve  greater  efficiencies  in  the  weapon  system  life  cycle. 
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Appendix  B  GoReader  Report 


IITA  Progress  Report 


Electronic  Textbooks 
for 

Digital  Technical  Orders 
Phase  I:  goReader 


Anthony  S.  Martinez,  Captain,  USAFR 
Institute  for  Information  Technology  Applications 


Ellen  M.  Fiebig,  Major,  USAF 
Institute  for  Information  Technology  Applications 


October  2001 
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INTRODUCTION 


Air  Force  Materiel  Command  (AFMC)  Digital  Technical  Order  (TO)  Architecture  and 
Transition  status  is  progressing  Air  Force  wide.  This  report  contains  data  collected  from 
testing  the  applicability  of  using  a  particular  “e-reader”  device  for  digital  technical  orders 
and  assorted  logistics  applications  in  a  field  environment.  The  ability  of  electronic  books 
(e-books),  palm  reader  (e-reader),  or  industrial  PC  laptop  users  to  apply  the  digital  TO 
data  to  daily  maintenance  or  other  logistical  operations  enhances  mission  performance. 
Ease  of  deployment,  interactivity  with  publication  data,  highlighting,  updates,  adjusting 
size  and  resolution,  and  luminosity  of  the  display  are  some  other  major  advantages. 

BACKGROUND 

T echnical  Order  warehouses  take  up  roughly  250,000  sq  ft  of  valuable  floor  space  and 
cost  $12M/year  to  operate.  A  typical  flying  squadron  dedicates  a  12x16  ft  room  to  the 
TO  library  and  spends  90  to  120  man  hours  per  month  maintaining  it.  MC-130H  aircraft 
carry  a  TO  library  that  weighs  270  lbs.  Airline  studies  show  approximately  20% 
reduction  in  turn  time  on  aircraft  maintenance.  (AFMC/DRR  brief,  13  Aug  01) 

The  majority  of  the  AF  TOs  have  already  begun  digitization.  Currently  all  major  Air 
Force  Commands  are  working  through  AFMC  to  ensure  timely  transition  to  useful 
technical  orders  for  active  and  reserve  maintenance  personnel.  The  concept  of  the 
Interactive  Electronic  Technical  Manual  (lETM)  was  developed  in  the  mid  1990’s,  and  is, 
at  present,  coordinated  jointly  with  USAF  Propulsion  contractors  Pratt  and  Whitney,  and 
General  Electric. 


RESEARCH  OBJECTIVE 

Determine  if  electronic  books  are  a  suitable  document  format  for  equipment  manuals? 
Can  they  support  weapon  system  schematics?  Could  this  be  an  improved  solution  for 
updates?  Can  these  devices  simplify  deployments? 

PROJECT  DESCRIPTION/METHODOLOGY 

Initially,  testing  began  by  seeking  digitization  status  of  Technical  Orders,  and  other  Air 
Force  Manuals  and  Publications.  Working  with  HQ  AFRC/LGA,  Robins  AFB,  Georgia,  it 
was  determined  that  the  best  weapons  system  program  office  to  gauge  progress  on  data 
digitization  for  use  on  an  “e-reader”  device  was  the  fighter  engine  management  office 
located  at  Wright  Patterson  AFB,  Ohio.  The  FI  10-GE-129  engine,  used  extensively  by 
active  duty  and  reserve  units  on  the  F-15  and  F-16  weapons  systems,  currently  has 
comprehensive  digital  data  that  is  applicable  to  testing  on  “e-readers.”  IITA  obtained  a 
copy  of  the  lETM  digital  TO  on  CD  ROM  and  tested  the  data  and  its  use  on  the 
goReader  device  with  the  following  maintenance  units: 

1 .  F-16  Logistics  Group  at  Buckley  ANGB 

2.  F-16  Logistics  Group  at  Shaw  AFB  SC 

3.  F-16  Logistics  Group  at  McEntire  ANGB,  SC 

A  survey  was  conducted  with  the  maintainers  in  each  unit  to  address  project 
objectives. 
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RESULTS 


Buckley  AFB,  ANG  F-16  Unit;  In  this  test,  researchers  presented  the  goReader  device 
and  TO  data  in  non-vector  format  for  discussion  and  review  by  maintenance  troops. 
Many  of  the  eniisted  engine  troops  were  thoroughiy  impressed  by  the  size  and  screen 
resoiution  of  the  goReader.  Questions  regarding  the  durabiiity  of  the  device  were 
brought  up  often,  and  data  was  ioaded  and  used  interactiveiy  in  a  non-shop 
environment.  The  unit  is  not  currentiy  aiigned  for  digitai  TO  use,  so  feedback  was  taken, 
but  operationai  testing  was  iimited. 

Shaw  AFB,  F-16  Unit:  The  28th  Wing  is  currentiy  the  “test  base”  for  USAF  digitai  TO 
depioyment.  The  Component  Repair  Squadron  Propuision  Shop  is  the  most  fuiiy 
capabie  at  computer  appiications,  testing,  and  actuaiiy  using  digitai  TO’s  on  reader 
devices.  The  goReader  device  was  ioaded  with  as  much  TO  data  as  possibie.  One 
weakness,  determined  in  the  fieid,  was  that  the  memory  capacity  of  the  goReader  was 
insufficient  for  the  entire  digitai  engine  TO.  Maintenance  troops  iiked  the  screen  size 
and  resoiution  of  the  data,  but  durabiiity  was  a  major  probiem.  in  moving  the  reader 
around,  often  times  the  data  wouid  surge  or  be  iost.  in  addition,  the  concept  of  using  a 
reader  with  a  iarger  data  memory  bank  or  even  CD  ROM  Reader  was  brought  up. 

McEntire  ANG. 

Currentiy,  this  is  the  oniy  Guard  Wing  to  utiiize  iETMs  on  shop  engine  maintenance. 

The  technician  we  interviewed  at  McEntire  ANG  was  thoroughiy  impressed  by  the 
technoiogy  and  upgrades  continuaiiy  being  made  to  the  technoiogy.  PC  Reader  was  the 
preferred  mode  of  reading  data;  however,  in  ebook  form,  the  data  was  usefui  as  iong  as 
no  scroiiing  or  data  searches  were  required.  The  technician  preferred  not  to  move  the 
reader,  and  instead,  consuited  it  when  data  or  schematic  information  was  necessary. 

CONCLUSION 


Objectives: 

Determine  if  eiectronic  books  are  a  suitabie  document  format  for  equipment  manuais? 

IITA  research  has  found  that  there  are  suitable  software  applications 
for  the  electronic  books  with  regards  to  digital  data  used  by  the  Air  Force.  Field 
Manuals,  Maintenance  Technical  Orders,  Supply/Logistics  inventories,  and  an 
array  of  other  applications  are  highly  feasible  with  reader  devices  and  durable  PC 
e-book  technology. 

Can  they  support  weapon  system  schematics? 

Data  on  Maintenance  Schematics  were  originally  scanned  and  programmed 
in  raster  or  “image  only”  format,  which  is  highly  viewable  on  e-reader  technology. 
However,  AFMC,  along  with  several  defense  contractors  are  working  to  make  the 
schematics  more  navigable  for  our  technicians  and  troops  by  converting  to  vector 
or  “scale  adjustable  images”  which  are  only  readable  on  e-readers  with  CPU 
capability  (i.e.  PC  notebooks,  or  their  hybrid  technologies). 
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Could  this  be  an  improved  solution  for  updates? 

Expanding  the  memory  capability  and  enhancing  the  durability  protection 
of  the  e-reader  or  GoReader  would  provide  one  alternative.  Also,  the  ability  to 
read  a  CD  ROM  either  with  the  e-reader  or  a  nearby  docking  station  would  be 
helpful  in  order  to  make  entire  digital  libraries  of  data  available  to  the  technician 
with  ease  of  access  and  convenience  of  usage. 

Can  these  devices  simplify  deployments? 

Yes,  without  question,  the  use  of  e-reader  devices  and  compatible  PC 
reader  technology  makes  deployments  easier.  There  are  not  as  many  data  library 
items  required  for  deployment,  and  the  data  is  more  conveniently  accessible.  We 
are  seeing  the  use  of  interactive  e-readers  and  durable  notebook  technology  used 
more  and  more  in  the  field. 


SURVEY 


Survey  Questions 

(goReader  and  AF  Digital  Technical  Orders) 


1 .  Could  the  goReader  device  accept  the  sample  digital  TOs? 

Yes,  the  goReader  device  could  accept  digital  TO’s.  However,  the  successful 
use  of  the  TO  currently  requires  either  a  CD  Rom  reader  capability,  or  extensive 
memory  beyond  the  15MB  currently  available  on  the  goReader. 

2.  Was  it  readable? 

Data  was  readable  in  PDF  Format  Only.  Other  Raster  and  Vector  versions  of  the 
digital  TO’s  are  not  readable  on  the  existing  goReader  software. 

3.  How  much  data  could  be  loaded?  (How  many  volumes  of  TOs  could  the  device 
load?) 

Currently  the  goReader  could  maintain  up  to  15  MB  of  TO's,  however,  the  data 
must  all  be  stored  on  goReader  data  cartridge  to  prevent  loss.  A  typical  TO  CD 
consists  of  up  to  640MB  of  data,  and  requires  Internet  access  at  times  to  access 
changes  and  new  TO  data. 

4.  Could  the  data  be  manipulated  in  any  way?  (Highlighted?  Note-taking? 
Printing?) 

The  goReader  allowed  for  Highlighting,  and  Note-taking  using  on-screen 
displays.  Printing  was  very  difficult  without  a  print  driver  or  compatible  PC  to 
mate  with  the  goReader.  When  data  files  were  shared  successfully  between  the 
goReader  and  the  PC’s,  the  data  was  fully  usable,  which  is  a  leading  result  for 
recommending  the  use  of  actual  PC  or  laptop  technology  made  durable  for 
industrial  or  field  use. 
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5.  How  much  battery-life? 

Very  poor... not  more  than  2  hours  extended  life  in  any  of  the  trials. 

6.  Is  it  durable  enough  for  flight  line  use  and  deployments? 

The  device  is  not  very  reliable...  While  we  did  not  expose  the  device  to  any 
condition  which  might  break  it,  we  did  shake  and  move  the  goReader  around. 
Many  times  resulting  in  complete  loss  of  data  and  requiring  re-boot.  Durability 
would  have  to  be  improved  before  flight  line  use  could  be  recommended. 

7.  What  is  the  process  for  updates? 

Currently,  TO  updates  are  made  on  the  Internet  by  the  TO  Distribution  Officer 
(TODO)  headquartered  at  Oklahoma  City  Air  Logistics  Center.  The  goReader 
requires  either  mating  or  a  direct  Ethernet  connection  to  get  the  Internet  updates, 
and  then  needs  to  be  mated  to  a  PC  or  laptop  in  order  to  use  the  full  CD  data  in 
accessing  the  file  downloads. 

8.  What  enhancements  would  you  like  to  see? 

Hybridization  in  product  types  between  the  goReader  and  a  typical  PC  would  be 
quite  useful.  Reliability,  with  processing  and  data  storage  power,  as  well  as 
durability  is  the  key  tenets  of  making  a  successful  device  that  can  be  used  by  the 
military.  Troops  likes  the  large  screen  display  and  touch  capability  of  the 
goReader,  but  data  retention  and  access  to  the  full  complement  of  Internet  and 
CD  based  data  were  a  major  shortcoming.  In  addition,  we  cannot  say  enough 
about  the  lack  of  durability  of  the  product. 

PHASE  II  PLAN 

Research  found  that  digital  TO  readers  would  be  more  successful  with  durable 
technology  associated  with  the  reader  such  as  the  Panasonic  “Toughbook,”  or  Dolch 
Computer  Systems  technology.  These  added  secure  data  features  reduce  the  loss  of 
data  in  the  industrial  or  field  environment  providing  reliability  greater  than  95%  for  data 
storage,  digital  display,  and  portability.  As  goReader  technology  evolves,  IITA  believes 
there  will  be  expanded  potential  usage  for  Dolch  hard  casing  in  order  to  preserve  data 
as  well. 

A  proposal  will  be  submitted  to  research  “ruggedized”  electronic  technology  on 
digital  TO  use.  IITA  is  seeking  partnership  with  AFRC/LGA. 
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Appendix  C  Survey 


IITA 


Institute  for  Information  Technology  Applications 
US  Air  Force  Academy 


DIGITAL  TECHNICAL  ORDERS  (T.O.)  EVALUATION  DATA 
USER  FEEDBACK 


BASE: _  DATE: _ 

NAME/RANK: _ EVALUATION  PERIOD: _ 

Introduction:  The  Panasonic  CF  28  Laptop  computers  are  part  of  a  larger  program 
being  evaluated  for  the  Air  Force.  Your  comments  and  opinions  are  of  extreme 
value,  thus,  we  thank  you  for  your  participation.  Please  feel  free  to  express  your 
opinion  about  the  computers  and  whether  or  not  you  think  it  is  a  viable 
replacement  for  paper-based  technical  orders.  The  results  of  this  survey  will  be 
available  in  a  technical  report  at  the  conclusion  of  this  project. 

PART  I 

Please  circle  your  responses  to  the  following  questions: 

1 .  How  often  do  you  use  the  digital  TO  in  your  current  work  environment? 


Never 

Sometimes 

About  Half  the 
Time 

Often 

Always 

2.  How  often  do  you  use  a  desktop  or  laptop  personal  computer? 


Never 

Sometimes 

About  Half  the 
Time 

Often 

Always 

3.  In  using  the  lETM  digital  TOs  on  your  unit’s  assigned  “Toughbook”  computer,  do  you 
ever  stumble  with  technology  issues  that  require  additional  training? 


Never 

Sometimes 

About  Half  the 
Time 

Often 

Always 

Please  describe 

the  problems  you  have  encountered  and  provide  specific  examples. 

(Use  the  back  of  this  survey  if  you  need  more  space.) 


4.  How  often  to  you  find  the  need  to  cross-reference  the  data  from  the  CD-ROM  digital 
TO  with  an  actual  printed  version  of  the  TO? 


Never 

Sometimes 

About  Half  the 
Time 

Often 

Always 
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5.  During  the  period  you  have  been  using  digital  TO’s,  have  you  had  any  Information 
Technology  Problems  with  the  “Toughbook”  hardware? 

Never  Sometimes  About  Half  the  Often  Always 

_ _ Time _ _ 

Please  describe  the  problems  you  have  encountered  and  provide  specific  examples. 


6.  Have  you  had  any  applications  problems  with  the  software  or  CD-ROM  Data? 


Never  Sometimes  About  Half  the  Often  Always 

_ _ Time _ _ 

Please  describe  the  problems  you  have  encountered  and  provide  specific  examples. 


PART  II. 

Please  circle  the  best  rating  for  each  of  the  following  items  and  provide  comments  as 
necessary: 


1.  QUALITY 

a.  Ease  of  Use 

Comments 

Poor 

Fair 

Good 

Excellent 

b. 

Picture  Quaiity 

Poor 

Fair 

Good 

Excellent 

Comments 

c. 

Readabiiity 

Poor 

Fair 

Good 

Excellent 

Comments 

d. 

Font  Quaiity 

Poor 

Fair 

Good 

Excellent 

Comments _ 

d.  Features  (finding,  indexing,  note  taking.  Unking) 

Poor  Fair  Good  Excellent 

Comments _ 

e.  Battery  Life  Poor  Fair  Good  Excellent 

What  was  the  actual  battery  life? _ 
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2.  PRACTICALITY 

a.  Time-Savings  Poor  Fair  Good  Excellent 

Comments _ 

b.  Rate  vs.  Paper  TO  Poor  Fair  Good  Excellent 


Comments 

c.  Portabiiity 

Poor 

Fair 

Good 

Excellent 

Comments 

d.  Reiiabiiity 

Poor 

Fair 

Good 

Excellent 

Comments 

e.  Durabie 

Poor 

Fair 

Good 

Excellent 

Comments 


PART  III.  WRITTEN  RESPONSES 

1.  On  average,  how  many  hours  per  day  did  you  use  the  Laptop? 

2.  Which  TO  data  files  did  you  utilize? 


5.  Please  give  us  your  overall  opinion  of  the  CF  28  Laptop. 
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6.  Other  comments: 
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APPENDIX  D 


Equipment  Description 

Equipment  tested  was  acquired  through  either  the  Air  Force  Reserve  or  vendor  test-loan 
programs. 


Panasonic  CF-27 


TOUGHBOOK  27 


RUGGEDIZED 

AND  WIRELESS 

Intel®  Pentium®  III  Processor  SOOMHz 
12.1"  LCD  with  Touchscreen 
Up  to  2CK3B  HDD 

Full  Magnesium  Alloy  Case  with  Carry  Handle 
Moisture-  and  Dust-resistant  Design 
Shock-mounted,  removable  HDD 
Wireless-ready  Design  wire  less' 

<7  REAOY 


Panasonic  CF-28 


TOUGHBOOK  28  POWERFULRUGGEDIZED 

r  AND  WIRELESS 


•  Full  Magnesium  Alloy  Case  with  Carry  Handle 

•  Moisture-  and  Dust-resistant  Design 

•  Shock-mounted,  Removable  HDD 

•  13.3”  Transmissive,  Anti-reflective  LCD 

•  12.1”  Transflective,  Daylight-readable  LCD 

•  Wireless-ready  Design 
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Panasonic  CF-34 


RUGGEDIZED, 

ULTRA-PORTABLE 

AND  WIRELESS 

•  3.8  lbs.  Ultra-portable  Design 

•  Full  Magnesium  Alloy  Case 

•  Moisture-  and  Dust-resistant  Design 

•  Hand/Shoulder  Strap  Standard 

•  Wireless-ready  Design 


WIRELESS 
READY  1 


Panasonic  MDWD-07 

TOUGHBOOK 


07  WITH  [MDWD] 

MOBILE  DATA  WIRELESS  DISPLAY 


•1.5  lb.  Wireless  Display  Works  up  to 
300  ft.  from  the  Toughbook  07  or  any 
LANA/VAN  Wired  Toughbook 


•  Transflective,  Daylight-readable  Touchscreen 

•  With  Optional  Holster,  the  Lightweight 
Toughbook  07  Provides  True  Mobility 

•  Moisture-  and  Dust-resistant  Casing  and  LCD 

•  Sealed  Port  and  Connector  Covers 
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GLOSSARY  OF  DIGITAL  TO  JET  ENGINE  MAINTENANCE  STANDARDS, 
DEFINITIONS  AND  TERMINOLOGY 


AFMC 

ANG 

AFR 

AFRC 

ARS 

CAMS 

C3IPS 

CPU 

DoD 

FMC 

FW 

G081 

HTML 

lETM 

IITA 

MC 

NMCM 

NMCS 

NMCB 

TNMCM 

TNMCS 

OC-ALC 

PCMA 

PDF 

PMC 

PMCM 

PMCB 

PMCS 

TO 

SME 

xTML 


Air  Force  Material  Command 

Air  National  Guard 

Air  Force  Reserve 

Air  Force  Reserve  Command 

Air  Reserve  Station 

Core  Automated  Maintenance  System 

Command,  Control  and  Communications  Information  Processing  System 

Central  Processing  Unit 

Department  of  Defense 

Fully  mission  capable 

Fighter  Wing 

Tanker/Cargo  aircraft  version  of  CAMS 
Hyper  Text  Markup  Language 
Interactive  Engine  Technical  Manual 
Institute  for  Information  Technology  Applications 
Mission  capable  rate 

Not  MC  for  maintenance,  unscheduled  or  scheduled  rate 
Not  MC  for  supply  rate 

Not  MC  for  maintenance  and  supply,  unscheduled  or  scheduled  rate 

Total  not  MC  for  maintenance  (NMCM  +  NMCB)  rate 

Total  not  MC  for  supply  (NMCS  +  NMCB)  rate 

Oklahoma  City-Air  Logistics  Center 

Power  Controlled  Multiple  Access 

Portable  Document  Format 

Partially  mission  capable 

PMC  for  maintenance  rate 

Partially  MC  both  maintenance  and  supply  rate 

PMC  for  supply  rate 

Technical  Order 

Subject  Matter  Expert 

Extensive  Telephony  Markup  Language 
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Glossary  of  Air  Force  Technical  Orders 


00-5-1 

00-20-1 

00-20-2 

00-20-5 

00-25-107 

00-25-254-1 

00-35D-54 


Air  Force  Technical  Order  System 
Aerospace  Equipment  Maintenance  General  Policy  and 
Procedures  (with  RAC  1  incorporated) 

Maintenance  Data  Documentation 

Aerospace  Vehicle  Inspection  and  Documentation 

Maintenance  Assistance 

Comprehensive  Engine  Management  System  Engine 

Configuration,  Status,  and  TCTO  Reporting  Procedures 
USAF  Deficiency  Reporting  and  Investigating  System 


End  Notes 


*  Deputy  Secretary  of  Defense  Memorandum,  “Policy  for  the  Transition  to  a  Digital  Environment  for 
Acquisition  Programs,  2  July  1997 

^  Air  Force  Technical  Order  Transformation,  Architecture/Transition  Roadmap,  Directorate  of 
Requirements,  HQ  Air  Force  Materiel  Command,  August  2002. 

^  Colonel  Thomas  Di  Nino,  HQ  AFMC/DRR,  PowerPoint  briefing  on  Air  Force  Technical  Order  Vision 
briefing  on  Architecture/Transition  Status,  slide  47  and  48,  13  Aug  2001. 

^  Air  Force  News,  “Air  Force  Going  Digital  with  Technical  Orders”,  by  Senior  Airman  Lee  Watts,  20* 
Fighter  Wing  Public  Affairs,  Shaw  AFB,  SC  (AFPN) 
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